Image Utility Assessment and a Relationship with Image Quality Assessment

International audiencePresent quality assessment (QA) algorithms aim to generate scores for natural images consistent with subjective scores for the quality assessment task. For the quality assessment task, human observers evaluate a natural image based on its perceptual resemblance to a reference. Natural images communicate useful information to humans, and this paper investigates the utility assessment task, where human observers evaluate the usefulness of a natural image as a surrogate for a reference. Current QA algorithms implicitly assess utility insofar as an image that exhibits strong perceptual resemblance to a reference is also of high utility. However, a perceived quality score is not a proxy for a perceived utility score: a decrease in perceived quality may not affect the perceived utility. Two experiments are conducted to investigate the relationship between the quality assessment and utility assessment tasks. The results from these experiments provide evidence that any algorithm optimized to predict perceived quality scores cannot immediately predict perceived utility scores. Several QA algorithms are evaluated in terms of their ability to predict subjective scores for the quality and utility assessment tasks. Among the QA algorithms evaluated, the visual information fidelity (VIF) criterion, which is frequently reported to provide the highest correlation with perceived quality, predicted both perceived quality and utility scores reasonably. The consistent performance of VIF for both the tasks raised suspicions in light of the evidence from the psychophysical experiments. A thorough analysis of VIF revealed that it artificially emphasizes evaluations at finer image scales (i.e., higher spatial frequencies) over those at coarser image scales (i.e., lower spatial frequencies). A modified implementation of VIF, denoted VIF*, is presented that provides statistically significant improvement over VIF for the quality assessment task and statistically worse performance for the utility assessment task. A novel utility assessment algorithm, referred to as the natural image contour evaluation (NICE), is introduced that conducts a comparison of the contours of a test image to those of a reference image across multiple image scales to score the test image. NICE demonstrates a viable departure from traditional QA algorithms that incorporate energy-based approaches and is capable of predicting perceived utility scores

Subjective Quality Evaluation of H.264 High-Definition Video Coding versus Spatial Up-Scaling and Interlacing

International audienceThe upcoming High-De nition format for video display provides high-quality content, especially when displayed on adapted devices. When combined with video coding techniques such as MPEG-4 AVC/H.264, the transmission of High-De nition video content on broadcast networks becomes possible. Nonetheless, transmitting and decoding such video content is a real challenge. Therefore, intermediate formats based on lower frame resolutions or interlaced coding are still provided to address targets with limited resources. Using these formats, the nal video quality depends on the postprocessing tools employed at the receiver to upsample and de-interlace these streams. In this paper, we compare the full-HD format to three possible scenarios to generate a full-HD stream from intermediate formats. We present the results of subjective tests that compare the visual quality of each scenario when using the same bitrate. The results show that using the same bitrate, the videos generated from lower-resolution formats reach similar quality compared to the full-HD videos

RELIABILITY OF 2D QUALITY ASSESSMENT METHODS FOR SYNTHESIZED VIEWS EVALUATION IN STEREOSCOPIC VIEWING CONDITIONS

International audienceThis paper investigates the reliability of objective quality metrics commonly used for the quality assessment of 2D media, in the context of 3D Video. In the absence of any dedicated tool for the evaluation of synthesized views quality, we often rely on available 2D metrics for direct evaluation of 3D media quality, or with some adaptation to the 3D case. However, recent studies showed that the use of DIBR, depending on its in-painting strategy, can lead to downsides whose range in terms of quality, has never been experienced with 2D media. This paper questions the reliability of the objective quality metrics normally used for the quality assessment, when assessing stereopairs. Seven DIBR algorithms are used to generate novel viewpoints. A series of commonly used quality metrics then assess their quality. The results of our experiments showed that the metrics are not sufficient to faithfully predict human judgment. Moreover, we compared our results with an experiment run in monoscopic viewing condition and the differences are for the less unexpected since the preferred artifacts in monoscopic condition are the most rejected in stereoscopic condition. This paper proposes some explanations

Influence of the source content and encoding configuration on the perceived quality for scalable video coding

International audienceIn video coding, it is commonly accepted that the encoding paramaters such as the quantization step-size have an influence on the perceived quality. When dealing with Scalable Video Coding (SVC), the parameters used to encode each layer logically have an influence on the overall perceived quality. It is also commonly accepted that using given encoding parameters, the perceived quality does not change significantly according to the encoded source content. In this paper, we evaluate the impact of both SVC coding artifacts and source contents on the quality perceived by human observers. We exploit the outcomes of two subjective experiments designed and conducted under standard conditions in order to provide reliable results. The two experiments are aligned on a common scale using a set of shared processed video sequences, resulting in a database containing the subjective scores for 60 different sources combined with 20 SVC scenarios. We analyse the performance of several source descriptors in modeling the relative behaviour of a given source content when compared to the average of other source contents

Can 3D synthesized views be reliably assessed through usual subjective and objective evaluation protocols?

International audienceThis paper addresses the problem of evaluating virtual view synthesized images in the multi-view video context. As a matter of fact, view synthesis brings new types of distortion. The question refers to the ability of the traditional used objective metrics to assess synthesized views quality, considering the new types of artifacts. The experiments conducted to determine their reliability consist in assessing seven different view synthesis algorithms. Subjective and objective measurements have been performed. Results show that the most commonly used objective metrics can be far from human judgment depending on the artifact to deal with

Perceived quality of DIBR-based synthesized views

International audienceThis paper considers the reliability of usual assessment methods when evaluating virtual synthesized views in the multi-view video context. Virtual views are generated from Depth Image Based Rendering (DIBR) algorithms. Because DIBR algorithms involve geometric transformations, new types of artifacts come up. The question regards the ability of commonly used methods to deal with such artifacts. This paper investigates how correlated usual metrics are to human judgment. The experiments consist in assessing seven different view synthesis algorithms by subjective and objective methods. Three different 3D video sequences are used in the tests. Resulting virtual synthesized sequences are assessed through objective metrics and subjective protocols. Results show that usual objective metrics can fail assessing synthesized views, in the sense of human judgment

Tradeoffs in Subjective Testing Methods for Image and Video Quality

International audienceThe merit of an objective quality estimator for either still images or video is gauged by its ability to accurately estimate the perceived quality scores of a collection of stimuli. Encounters with radically different distortion types that arise in novel media representations require that researchers collect perceived quality scores representative of these new distortions to confidently evaluate a candidate objective quality estimator. Two common methods used to collect perceived quality scores are absolute categorical rating (ACR)1 and subjective assessment for video quality (SAMVIQ).2,3 The choice of a particular test method affects the accuracy and reliability of the data collected. An awareness of the potential benefits and/or costs attributed to the ACR and SAMVIQ test methods can guide researchers to choose the more suitable method for a particular application. This paper investigates the tradeoffs of these two subjective testing methods using three different subjective databases that have scores corresponding to each method. The subjective databases contain either still-images or video sequences. This paper has the following organization: Section 2 summarizes the two test methods compared in this paper, ACR and SAMVIQ. Section 3 summarizes the content of the three subjective databases used to evaluate the two test methods. An analysis of the ACR and SAMVIQ test methods is presented in Section 4. Section 5 concludes this paper